Sheet music creation method and image processing system

ABSTRACT

Image data of sheet music formed from a plurality of pages is input, and the input image data is divided into bars. The complexity of performance of each divided bar is determined for each page. An easy-to-play portion is detected as a page feed position for each page in accordance with the determination result. The layout of the sheet music is changed based on the detected page feed position, and the layout-changed sheet music is output.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a sheet music creation method and imageprocessing system and, more particularly, to a sheet music creationmethod and image processing system for analyzing image data of sheetmusic input from an image input device such as a scanner ormultifunctional peripheral, and changing the layout of the sheet musicin accordance with the analysis result.

2. Description of the Related Art

As low-cost scanner devices become available, an original can be easilyscanned and converted into digital data even at home. Inkjet printers asprinting apparatuses have been developed, and even a multifunctionalperipheral called a multi-function printer capable of not only printingan image, but also scanning an image becomes popular. The user can use ascanner device and printer device to easily copy his originals andcontents at home.

An example of contents which improve user's convenience when copied is“sheet music”. In general, the user purchases sheet music as a book.However, it impairs user's convenience to set sheet music purchased as abook directly on a music stand. For example, when the user sets sheetmusic of many pages directly on a music stand to see a desired page, hemust forcibly open the sheet music, damaging the sheet music. To preventthis, a music stand devised to fix sheet music at a desired page is alsocommercially available. However, such a music stand makes it difficultto turn a page. It is generally cumbersome to turn a page of sheet musicduring the performance. This work is very troublesome especially duringthe performance of a musical instrument using both hands, such as thepiano.

During the music lesson of the piano or the like, the teacher oftenwrites an instruction directly on sheet music. It is popular to copysheet music and keep the original clean.

When editing and distributing sheet music of a composition by a user orwhen editing music composed in response to user's request anddistributing it as sheet music, various editing operations are required.For this purpose, a technique for, for example, editing sheet music isdisclosed.

Conventionally, when editing sheet music, the player finds out from it aportion where he can easily turn a page, and edits the sheet music sothat he can turn a page at this portion. However, according to thissheet music editing method, the player sometimes wrongly edits sheetmusic. A long time is taken to create new sheet music. To prevent this,the following technique is disclosed.

Japanese Patent Laid-Open No. 2003-177745 discloses an image processingapparatus which allows the user to set a page feed sign at a desiredposition in digital data of sheet music, and turn a page uponrecognizing the sign. This image processing apparatus allows turning apage in accordance with the level of the performance technique of aplayer or his preference. Japanese Patent Registration No. 3801939discloses an image processing apparatus which turns a page bydetermining, as a break of sheet music, a position at which apredetermined number of specific characters or signs continue. Thisimage processing apparatus can automatically turn a page.

SUMMARY OF THE INVENTION

Sheet music as a general book describes information for a performance onthe basis of rules built up through the historical development. Sheetmusic is not convenient for all players. A player edits sheet music by,for example, copying it and dividing the copy into a plurality of piecesso as to easily turn a page, or coloring sheet music or adding signs toit so as to make it easy for himself to play. The editing work optimizessheet music for the player. However, it is very cumbersome for theplayer to edit sheet music, so demands have arisen for easily editingsheet music.

However, according to the image processing apparatus in Japanese PatentLaid-Open No. 2003-177745, the user must set a page feed position ofsheet music. When sheet music has a large number of pages, it is verycumbersome to set page feed positions in all pages. The image processingapparatus in Japanese Patent Registration No. 3801939 determines, as abreak of sheet music, a position at which a predetermined number ofspecific characters or signs continue. However, if no such positionexists, the image processing apparatus cannot set a page feed position.

The present provides a sheet music creation method and image processingsystem capable of easily, reliably creating sheet music for personal usethat facilitates the turn of pages.

The present invention in its first aspect provides a sheet musiccreation method of creating new sheet music by changing a page feedposition in sheet music formed from a plurality of pages. The methodincludes inputting image data of the sheet music formed from theplurality of pages; dividing the image data input in the step ofinputting into bars; determining, for each page, complexity ofperformance of each bar divided in the steps of dividing; detecting aneasy-to-play portion as a page feed position for each page in accordancewith a determination result in the step of determining; changing alayout of the sheet music on the basis of the page feed positiondetected in the step of detecting; and outputting the sheet music whoselayout has been changed in the step of changing.

The present invention in its second aspect provides an image processingsystem which includes a host apparatus; an image reading apparatusconfigured to be connected to the host apparatus; and a printingapparatus configured to be connected to the host apparatus. The systemis configured to create new sheet music by changing a page feed positionin sheet music formed from a plurality of pages. The image readingapparatus reads image data of the sheet music formed from the pluralityof pages. The host apparatus includes a division unit configured todivide the image data read by the image reading apparatus into bars; adetermination unit configured to determine, for each page, complexity ofperformance of each bar divided by the division unit; a page feedposition detection unit configured to detect an easy-to-play portion asa page feed position for each page in accordance with a determinationresult of the determination unit; and a layout change unit configured tochange a layout of the sheet music on the basis of the page feedposition detected by the page feed position detection unit. Further, theprinting apparatus prints the sheet music whose layout has been changedby the layout change unit.

The present invention in a third aspect provides an image processingsystem including a host apparatus; and a multifunctional peripheralconfigured to be connected to the host apparatus. The multifunctionalperipheral includes an image reading unit; and a printing unit. Thesystem is configured to create new sheet music by changing a page feedposition in sheet music formed from a plurality of pages. The hostapparatus includes a division unit configured to divide, into bars,image data of sheet music which is read by the image reading unit andformed from a plurality of pages; a determination unit configured todetermine, for each page, complexity of performance of each bar dividedby the division unit; a page feed position detection unit configured todetect an easy-to-play portion as a page feed position for each page inaccordance with a determination result of the determination unit; and alayout change unit configured to change a layout of the sheet music onthe basis of the page feed position detected by the page feed positiondetection unit. The multifunctional peripheral is configured to causethe printing unit to print the sheet music whose layout has been changedby the layout change unit.

The present invention in a fourth aspect provides a computer-readablemedium containing computer-executable instructions to be utilized in animage processing system for creating new sheet music by changing a pagefeed position in sheet music formed from a plurality of pages. Inparticular, the medium includes computer-executable instructions forinputting image data of the sheet music formed from the plurality ofpages; computer-executable instructions for dividing the input imagedata into bars; computer-executable instructions for determining, foreach page, complexity of performance of each divided bar;computer-executable instructions for detecting an easy-to-play portionas a page feed position for each page in accordance with a determinationresult from the determining step; computer-executable instructions forchanging a layout of the sheet music on the basis of a detected pagefeed position; and computer-executable instructions for outputting thesheet music whose layout has been changed.

According to the aforementioned present invention, the complexity ofperformance is determined by analyzing image data of sheet music, andthe page feed position is changed to create new sheet music. Hence,sheet music which facilitates the turn of pages can be created. Sincethe page feed position is set automatically, the burden on the user isreduced. Further, image data is analyzed for each page to set a pagefeed position, so the page feed can be set reliably.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a conceptual view showing an example of the configuration ofan image processing system according to the present invention;

FIG. 2 is a block diagram of the image processing system in FIG. 1;

FIG. 3 is a schematic perspective view of the outer appearance of an MFPaccording to an embodiment of the present invention;

FIG. 4 is a block diagram of the MFP shown in FIG. 3;

FIG. 5 is a flowchart of image processing executed in copying;

FIG. 6 is a block diagram showing the basic configuration of an imageprocessing system according to the present invention;

FIG. 7 is a flowchart showing a processing sequence in the imageprocessing system having the configuration shown in FIG. 6;

FIG. 8 is a flowchart showing a processing sequence to convert imagedata into a proper format when analyzing the image data;

FIG. 9 is a view for explaining skew correction of an original image;

FIG. 10 is a view for explaining great staff block division;

FIG. 11 is a view for explaining bar block division;

FIG. 12 is a view for explaining a position suitable for a page feedposition;

FIG. 13 is a flowchart showing a concrete processing sequence of pagefeed position detection processing;

FIG. 14 is a flowchart showing a concrete processing sequence of pagefeed position detection processing;

FIG. 15 is a flowchart showing a concrete processing sequence of layoutchange processing;

FIG. 16 is an explanatory view showing examples of layout change; and

FIG. 17 is a flowchart showing a concrete processing sequence of pagefeed position detection processing.

DESCRIPTION OF THE EMBODIMENTS

Various embodiments of the present invention will be described in detailbelow with reference to the accompanying drawings. The same referencenumerals denote the same parts, and a description thereof will not berepeated.

An embodiment for practicing the present invention by applicationsoftware running on a personal computer (to be referred to as a PChereinafter) serving as a host apparatus will be explained.

FIG. 1 is a conceptual view showing an example of the configuration ofan image processing system according to the present invention.

An image processing system 101 in FIG. 1 includes a scanner 102 servingas an image reading apparatus, a PC 103, and a printer 104 serving as aprinting apparatus. It is possible to change the layout of sheet musicscanned by the scanner 102 and print the resultant sheet music by theprinter 104. The PC 103 is connected to an external network 105.

FIG. 2 is a block diagram of the PC 103 of the image processing system101 shown in FIG. 1.

In FIG. 2, reference numeral 201 denotes a ROM (Read Only Memory).Reference numeral 202 denotes a CPU (Central Processing Unit). Referencenumeral 203 denotes a RAM (Random Access Memory). Reference numeral 204denotes an I/O (Input/Output). Reference numeral 205 denotes a NIC(Network Interface Card). Reference numeral 206 denotes an HDD (HardDisk Drive). Reference numeral 207 denotes a bus.

The CPU 202 controls the operation of the overall host apparatus (PC103) via the bus 207 in accordance with a program and data stored in theROM 201. At the same time, the CPU 202 performs image processing usingthe RAM 203 as a work memory. For example, the CPU 202 performs imageprocessing for image data input via the I/O unit 204 or NIC 205, orimage data stored in advance in a storage medium such as the HDD 206.The CPU 202 outputs the processed image data via the I/O unit 204 or NIC205, or stores it in a storage medium such as the HDD 206.

For example, the I/O unit 204 is connected via a predetermined interfaceto an image input/output device such as a monitor (e.g., CRT or LCD), aprinter, or a scanner, or a storage device having a storage medium suchas a magnetic disk or optical disk. The I/O unit 204 can receive/outputimage data via the NIC 205 from/to a computer device connected to thestorage device and the above-mentioned image input/output device.Examples of the network are Ethernet (registered trademark), FDDI (FiberDistributed Data Interface), IEEE1394 serial bus, and USB (UniversalSerial Bus).

The same effects as the above-described ones can also be obtained byusing an image processing system in which a single image processingapparatus operates without the mediacy of a PC.

FIG. 3 is a schematic perspective view of the outer appearance of amulti-function printer (to be referred to as an MFP hereinafter) 301serving as a multifunctional peripheral according to an embodiment ofthe present invention. The MFP 301 has the function of a general PCprinter which receives data from a PC and prints it, and the function ofa scanner. Further, the MFP 301 has, as a function executed by a singleMFP, a copy function of printing, by a printer, an image scanned by ascanner, a function of directly reading image data stored in a memorycard or the like and printing it, or a function of receiving image datafrom a digital camera and printing it.

In FIG. 3, the MFP 301 includes a reading unit 302 such as a flat-bedscanner, a printing unit 303 of an inkjet type, electrophotography, orthe like, a display unit 304 serving as a display panel, and anoperation unit 305 serving as an operation panel having various keys,switches, and the like. A USB port (not shown) is provided on the rearsurface of the MFP 301 as a PC interface for communicating with a PC,and enables communication with a PC. In addition to these components,the MFP 301 includes a card interface 306 serving as a card slot forreading out data from a variety of memory cards, and a camera interface307 serving as a camera port for communicating data with a digitalcamera. Further, the MFP 301 includes an auto document feeder (to bereferred to as an ADF hereinafter) 308 for automatically setting anoriginal onto the original table.

FIG. 4 is a block diagram of the image processing system 101 shown inFIG. 3.

In FIG. 4, a CPU 401 controls various functions of the MFP, and executesan image processing program stored in a ROM 406 in accordance with apredetermined operation to the operation unit 305.

The reading unit 302 having a CCD reads an original image to outputanalog luminance data of red (R), green (G), and blue (B). The readingunit 302 may also include a contact image sensor (CIS) instead of theCCD. When the ADF 308 as shown in FIG. 3 is provided, the reading unit302 can successively read the original images of order sheets set on theADF 308.

The card interface 306 receives, in accordance with a predeterminedoperation to the operation unit 305, image data captured by a digitalcamera or the like and recorded on a memory card or the like. Ifnecessary, an image processing unit 402 converts image data received viathe card interface 306. For example, the image processing unit 402converts image data corresponding to the color space (e.g., YCbCr) of adigital camera into one corresponding to the RGB color space (e.g.,NTSC-RGB or sRGB). If necessary, the received image data undergoesvarious processes necessary for an application, such as resolutionconversion into an effective pixel count on the basis of headerinformation of the image data.

A camera interface 413 is directly connected to a digital camera to readimage data.

The image processing unit 402 performs image processes (to be describedlater) such as image analysis, calculation of the conversioncharacteristic, conversion from a luminance signal (RGB) into a densitysignal (CMYK), scaling, gamma conversion, and error diffusion. Dataobtained by these image processes are stored in a RAM 407. Whencorrected data stored in the RAM 407 reach a predetermined amountnecessary to print by the printing unit 303, the printing unit 303executes a print operation.

A nonvolatile RAM 408 is, for example, a battery backup SRAM, and storesdata unique to the MFP and the like.

The operation unit 305 has a photo direct print start key to selectimage data stored in a storage medium and start printing, a key to printan order sheet, and a key to read an order sheet. The operation unit 305also has a copy start key to perform monochrome copying or colorcopying, a mode key to designate a mode such as resolution or imagequality in copying, a stop key to stop a copy operation or the like, anumeric keypad to input the number of copies, a registration key, andthe like. The CPU 401 detects the pressed states of these keys, andcontrols each unit in accordance with the states.

The display unit 304 includes a dot matrix type liquid crystal display(LCD) and LCD driver, and presents various displays under the control ofthe CPU 401. The display unit 304 displays the thumbnail of image datastored in a storage medium.

The printing unit 303 includes an inkjet head, general-purpose IC, andthe like. The printing unit 303 reads out print data stored in the RAM407, and prints it out as a hard copy under the control of the CPU 401.

A driving unit 411 includes a stepping motor for driving feed anddelivery rollers in the operations of the reading unit 302 and printingunit 303, a gear for transmitting the driving force of the steppingmotor, and a driver circuit for controlling the stepping motor.

A sensor unit 410 includes a sensor for detecting the width of a printmedium, a sensor for detecting the presence/absence of a print medium, asensor for detecting the width of an original image, a sensor fordetecting the presence/absence of an original image, and a sensor fordetecting the type of print medium. Based on pieces of informationobtained from these sensors, the CPU 401 detects the states of anoriginal and print medium.

A PC interface 414 is an interface between the PC and the MFP. The MFPexecutes operations such as printing and scanning on the basis ofinstructions transmitted from the PC via the PC interface 414.

In copying, image data of an original image read by the reading unit 302is processed in the MFP, and printed by the printing unit 303. When theuser designates a copy operation via the operation unit 305, the readingunit 302 reads an original set on the original table. Image data of theread original image is transmitted to the image processing unit 402, andundergoes image processing (to be described later). The processed imagedata is transmitted to the printing unit 303, and printed.

FIG. 5 is a flowchart of image processing executed in copying.Respective steps will be explained, but a detailed description of stepswhich are not essential to the present invention will be omitted.

In step S110, image data which is read by the reading unit 302 andA/D-converted undergoes shading correction to correct variations ofimage sensors.

In step S120, the image data undergoes input device color conversion toconvert image data corresponding to a color space unique to the readingunit 302 serving as an input device into image data corresponding to astandard color space. More specifically, the image data is convertedinto image data corresponding to a color space such as sRGB defined byIEC (International Electrotechnical Commission) or AdobeRGB proposed byAdobe Systems. The conversion method is, for example, an arithmeticmethod based on a 3×3 or 3×9 matrix, or a look-up table method oflooking up to a table describing a conversion rule and determining acolor space on the basis of the table.

In step S130, the image data having undergone color conversion undergoesimage correction/manipulation processing. The processing contentsinclude edge emphasis processing of correcting blurring generated uponreading an original image, processing of removing offset generated uponreading by light irradiation, and character manipulation processing forimproving character readability.

In step S140, enlargement/reduction processing is executed to convertthe image data at a desired ratio when the user designates resizing orin layout copying of laying out two original sheets on one paper sheet.The conversion method is generally a bicubic method, nearest neighbormethod, or the like. When laying out and printing a plurality of imageson one print medium in layout copying or the like, the operations insteps S110 to S140 are repeated to read a plurality of images and layout the read images on one page. Then, the process shifts to thefollowing print operation.

In step S150, the image data corresponding to the standard color spaceis converted into image data corresponding to a color space unique to anoutput device. Similar to step S120, the conversion method suffices tobe the arithmetic method based on a matrix or the look-up table method.Image data corresponding to the color space unique to the output deviceis converted into one corresponding to the color space of the colors ofinks used in an inkjet MFP, such as cyan, magenta, yellow, and black.

In step S160, the image data undergoes quantization processing. Forexample, when an ink dot is expressed by a binary value representingwhether or not to discharge an ink dot, the image data suffices to bebinarized according to a quantization method such as error diffusion. Asa result, the image data is converted into a data format printable bythe printing unit. The print operation is executed based on the imagedata of this data format, forming an image.

A first embodiment of the present invention will be described belowreferring to the drawings. FIG. 6 is a block diagram showing the basicconfiguration of an image processing system according to the presentinvention.

The image processing system in FIG. 6 includes a sheet music data inputunit 601 which inputs digital data (sheet music data) of sheet music,and a block division unit 602 which divides sheet music data intoblocks. The image processing system also includes a page feed positiondetection unit 603 which detects a page feed position by analyzing sheetmusic data input by the sheet music data input unit 601. The imageprocessing system includes a layout change unit 604 which changes thelayout of sheet music data on the basis of a page feed position set bythe page feed position detection unit. Further, the image processingsystem includes a sheet music data output unit 605 which outputs newsheet music data whose layout is changed by the layout change unit 604.

The sheet music data input unit 601 may also be a unit which scans sheetmusic by a scanner, digitizes it, and outputs the digital data, or aunit which receives sheet music data via a network. The sheet music datainput unit 601 also has a unit which determines whether input sheetmusic data is a page in which the page feed position should be set. Thesheet music data input unit 601 may also have a unit capable of settingwhich of the right and left hands of a player for input sheet music ishis dominant hand.

The block division unit 602 divides sheet music data input from thesheet music data input unit 601 into blocks, and numbers the respectiveblocks. There are two types of unit blocks: one is a great staff, andthe other is a bar. Both great staff blocks and bar blocks are numbered.

The page feed position detection unit 603 divides, into bars, a page inwhich the page feed position should be detected, out of sheet music datainput from the sheet music data input unit 601, and analyzes sheet musicdata of each bar. The page feed position detection unit 603 detects abar having a lowest ratio of pixels printed per unit area from theimages of bars expressed by sheet music data. Then, the page feedposition detection unit 603 changes the page feed position. A bar havinga lowest ratio of pixels printed per unit area is sometimes positionednear the center of the page. However, this bar is not suitable for apage feed position, so the user may also designate the analysis range ofsheet music data. For example, when the user designates analysis of onlybars included in a great staff positioned at the bottom of a page, thepage feed position is reliably detected from the final great staff. Whenthe sheet music data input unit 601 sets the dominant hand of a player,it is also possible to analyze a great staff played by the dominant handand detect a page feed position. More specifically, the player plays thepiano with his both hands, sheet music is generally formed from greatstaffs. When the dominant hand of the player is the right hand, onlystaffs played by the right hand are analyzed to detect a page feedposition. In this case, the player can more easily turn a page with hisright hand which is his dominant hand.

The layout change unit 604 changes the layout of sheet music data inaccordance with a page feed position detected by the page feed positiondetection unit 603. As the layout change method, sheet music after apage feed position is cut out from a page set to change a page feedposition, and is appended to the beginning of the next page. To furthereasily turn a page, it is also possible to add one bar at the beginningof the next page to a margin at the bottom of the preceding page aftersheet music after a page feed position is appended to the beginning ofthe next page. As the addition method, the size or color of an added barmay also be changed to change its form from that of the remaining barsin order to easily discriminate the added bar from the remaining ones.

The sheet music data output unit 605 may also be a unit which prints outsheet music data on a print medium, or a unit which distributes it via anetwork.

FIG. 7 shows a sheet music creation processing sequence executed in theimage processing system having the configuration shown in FIG. 6. FIG. 7shows the basic operation of image processing by the image processingsystem.

In step S210, various settings are input. For example, a setting ofwhether to continuously scan an original of a plurality of pages, asetting of whether the first page is to be turned when continuouslyscanning an original, and a page setting of whether a page is to beturned when not continuously scanning an original are input. At thistime, it is preferable to store, in a memory, a page set to be turned,so as to discriminate it when performing subsequent processing. Someoriginals are formed from great staffs each made up of two staffs, likepiano sheet music, so the dominant hand may also be set. Further, agreat staff in which the page feed position is detected may also be set.

In step S220, all data are read from the sheet music data input unit. Byinput profile conversion, image data serving as sheet music data isconverted from an RGB signal obtained based on the spectralcharacteristic of a device into a signal corresponding to a standardcolor space where image processing is done.

In step S230, the image data read in step S220 is stored for each pagein the memory.

In step S240, the image data stored in the memory in step S230 undergoespre-processing and is converted into a format appropriate for analyzingimage data. Concrete contents of this processing will be describedlater.

In step S250, the image data converted in step S240 undergoes page feedposition detection processing to detect a page feed position. Concretecontents of this processing will be described later.

In step S260, the layout is changed by layout change processing on thebasis of the page feed position detected in step S250. Concrete contentsof this processing will be described later.

In step S270, the image data whose layout has been changed in step S260is distributed via a network, or the printer or the like prints based onthis image data.

Concrete processing in step S240 of FIG. 7 will be explained withreference to the flowchart of FIG. 8.

In step S310, skew correction of an original image is executed. This isbecause the reading device may read an original image with a skew. Skewcorrection suffices to use a method of correcting a skew using the staffof sheet music. For example, the image density of read sheet music iscalculated at a plurality of angles to create a histogram, as shown inFIG. 9. When the angle is parallel to the staff, the histogram drawssteep curves at portions corresponding to the five lines of the staff.From this, an angle at which the staff becomes parallel to the readingdirection is calculated, and the entire image is rotated and corrected.If necessary, a binarization operation may also be done using athreshold. Although a fixed threshold may also be used, the thresholdmay also be calculated after creating a brightness histogram when thehighest image density of an original image changes between pages or theimage density of a margin on an original changes between pages.Filtering processing may also be done to thicken the thin lines of astaff and hook, improving their reproducibility. The reference positionof the original image is determined using staff information in theskew-corrected image data. The interval represented by the staff isdifferent between G and F clefs, and thus G and F clefs are detected.Since G and F clefs are described at the beginning of a bar, theyundergo pattern matching with G and F clefs stored in advance in the ROMon the basis of the position information.

In step S320, the original image is divided into great staff blocks, andthe respective great staffs are numbered as shown in FIG. 10. This isbecause page feed position detection processing (to be described later)is done for each bar included in a designated great staff. Great staffblock division is performed by recognizing G or F clefs described at thebeginning.

In step S330, similar to the great staff block division, each greatstaff is divided into bar blocks, and the respective bars are numbered.FIG. 11 shows numbered bars (bars 1, 2, 3, 4, and 5) included in a givenstaff. Bars may also be numbered sequentially from the first barincluded in a great staff, as shown in FIG. 11, or from the first bar ofa page. Bar block division is performed by recognizing the bar lines ofbars. At this time, each bar cut out by block division is recognized asan image. For example, for bar 5 in FIG. 11, an image size of M pixels×Npixels is stored in the RAM in order to analyze the image of each bar insubsequent page feed position detection processing.

Finally in step S340, signs and rests are detected. As the sign and restdetection method, signs and rests are collated with those stored inadvance in the ROM by using pattern matching or the like, therebydetecting signs and rests included in each bar. The positions (e.g.,great staff numbers and bar numbers) and types of signs and rests arewritten in the RAM. Processing of recognizing and detecting signs andthe like on sheet music can adopt a well-known method, so a descriptionthereof will be omitted.

A position suitable for a page feed position will be explained. Inshort, the suitable position is an easy-to-play portion. Theeasy-to-play portion is a portion where the number of notes in sheetmusic is small. This will be explained with reference to FIG. 12. Assumethat a designated great staff of a given page includes bars 1201 to 1205in sheet music data input from the sheet music data input unit 601. Ofthese five bars, a bar which is easiest to play is the bar 1202 having asmallest number of notes. When each bar is recognized as an image fornumerical analysis, the number of notes in the bar 1202 is smallest andthe ratio of pixels printed per unit area is lowest among the remainingbars. That is, if each bar can be recognized as an image to obtain theratio of pixels printed per unit area, a bar having a low ratio ofpixels printed per unit area can be specified and detected as a pagefeed position.

Note that sheet music includes not only notes but also signs and restssuch as a sharp and flat. When obtaining the ratio of pixels printed perunit area in a bar including signs and rests, the ratio of pixelsprinted per unit area is finally attained by excluding the printedpixels of signs and rests. Whether a bar to be analyzed includes a signor rest can be determined by collating the position and type of sign orrest detected by the above-described sign and rest detection processingin step S340. As a result, the ratio of pixels printed per unit area ina bar can be attained.

Since the length of the staff is different between bars, the pixels ofthe staff need to be excluded in all bars when obtaining the ratio ofpixels printed per unit area in each bar.

Concrete processing of the page feed position detection processing instep S250 of FIG. 7 will be explained with reference to the flowchart ofFIG. 13.

In step S1010, sheet music data of one page having undergone thepre-processing in step S240 of FIG. 7 is acquired.

In step S1020, it is determined based on the setting in step S210 ofFIG. 7 whether the sheet music data acquired in step S1010 is a page inwhich the page feed position should be detected. If it is determinedthat the sheet music data is a page in which the page feed positionshould be detected, the process advances to step S1060. If it isdetermined that the sheet music data is not a page in which the pagefeed position should be detected, the process advances to step S1280.

In step S1060, it is determined whether a great staff to be analyzed isone designated in step S210 of FIG. 7. If it is determined that a greatstaff to be analyzed is a designated one, the process advances to stepS1070. If it is determined that a great staff to be analyzed is not adesignated one, the process advances to step S1240.

In step S1070, the number of pixels printed in a bar of interest in thegreat staff determined in step S1060 to be a designated one is obtained.

In step S1080, it is determined whether the bar of interest includes asign or rest. If the bar includes a sign or rest, the process advancesto step S1090. If the bar includes neither a sign nor rest, the processadvances to step S1100.

In step S1090, the number of pixels of the sign or rest is subtractedfrom the number of pixels printed in the bar of interest that isobtained in step S1070.

In step S1100, the number of pixels of the staff is subtracted from thenumber of pixels printed in the bar of interest.

In step S1110, the number of pixels printed in the bar of interest thatis obtained in step S1100 is divided by the area (e.g., M pixels×Npixels for bar 5 in FIG. 11) of the image range of the bar, obtainingthe ratio of pixels printed per unit area.

In step S1120, it is determined whether the bar for which the ratio ofpixels printed per unit area has been obtained is the final bar includedin the great staff. If the current bar is the final one, the processadvances to step S1240; if it is not the final one, to step S1230. Instep S1230, steps S1070 to S1120 are repeated for the next bar. In stepS1240, the next great staff is detected, and the process advances tostep S1250.

In step S1250, it is determined whether the current great staff is thefinal one in the page. If the current great staff is the final one, theprocess advances to step S1260; if it is not the final one, returns tostep S1060.

In step S1260, a bar having a lowest ratio of pixels printed per unitarea is obtained, and its bar number is stored in a memory. A positionimmediately after a bar of the bar number stored in the memory is set asa page feed position.

In step S1280, it is determined whether the sheet music is the finalone. If the sheet music data is not the final one, the process advancesto the next page in step S1290; if it is the final one, the page feedposition detection processing ends.

Page feed position detection processing when a dominant hand is set instep S210 of FIG. 7 will be explained with reference to FIG. 14. Stepsup to step S1020, and step S1060 and subsequent steps are the same asthose in FIG. 13, and a description thereof will not be repeated.

If it is determined in step S1020 that sheet music data is a page inwhich the page feed position should be detected, the process advances tostep S1030 to determine the dominant hand setting. If it is determinedthat the left hand is set as a dominant hand, the process advances tostep S1050 to extract staffs with F clefs, and then to step S1060. If itis determined that the right hand is set as a dominant hand, the processadvances to step S1040 to extract staffs with G clefs, and then to stepS1060.

More specifically, when the right hand is set as a dominant hand, onlystaffs with G clefs that are played by the right hand are extracted andanalyzed. When the left hand is set as a dominant hand, only staffs withF clefs that are played by the left hand are extracted and analyzed. Inthis case, the player can more smoothly turn a page in accordance withhis dominant hand. However, staffs with G clefs are not always played bythe right hand, and staffs with F clefs are not always played by theleft hand. For this reason, when setting a dominant hand, it ispreferable to be able to set even the position of a staff played by thedominant hand.

Concrete processing of the layout change processing in step S260 of FIG.7 will be explained with reference to the flowchart of FIG. 15. Assumethat a detected page feed position exists in the second half of a page.

In step S1510, sheet music data of one page is read.

In step S1520, it is determined whether the sheet music data read instep S1510 includes a page feed position. If the read sheet music dataincludes a page feed position, the process advances to step S1530; if itdoes not include a page feed position, to step S1550. Whether sheetmusic data includes a page feed position can be determined based on datastored in the memory in the page feed position detection processing instep S250 of FIG. 7.

In step S1530, the page feed position is recognized, and data of allbars after the page feed position are stored in a memory. The data aredeleted from the sheet music data read in step S1510.

In step S1540, data of the first bar out of data of respective barsstored in the memory area in step S1530 is read from the memory. Thedata is added to the end of the page so as to present this bar. Theplayer can check in advance the first bar of the next page that needs tobe memorized for turn of a page. As the bar addition method, it is alsopossible to downsize the added bar to become slightly smaller than theremaining ones, or change the color, as represented by 16A in FIG. 16.

In step S1550, if there is data after the page feed position in sheetmusic data of the preceding page stored in the memory, the data is read.The data is so added as to add sheet music of a part expressed by thedata to the beginning of a page and represent it.

In step S1560, the sheet music data edited in step S1540 or S1550 iscorrected to uniformly lay out great staffs in the page.

In step S1570, the sheet music data corrected in step S1560 is stored inthe memory.

In step S1580, it is determined whether all sheet music data have beenprocessed. If the above-described processes have not ended for all sheetmusic data, the process returns to step S1510 to repeat the processes instep S1510 to S1570 for sheet music data of the next page. If theabove-described processes have ended for all sheet music data, thelayout change processing ends.

When a detected page feed position exists in the second half of a page,no problem arises in the process of step S1550. However, when a detectedpage feed position exists in the first half of a page, sheet music dataafter the page feed position may not be fit in one page to which thedata are added. In this case, sheet music data before the page feedposition may also be added to the end of a page immediately preceding tothe page in which the page feed position is detected, as represented by16B of FIG. 16. To further smoothly turn a page, the same sheet musicdata as that added to the end of a preceding page may also be added in afaint color to the beginning of a page in which the page feed positionis detected, as represented by 16C of FIG. 16, in addition to theprocess represented by 16B of FIG. 16. The player can check the finalpart of a preceding page even after turning a page, and can smoothlyturn a page.

In the first embodiment, the complexity of performance of each bar isdetermined by obtaining, for each bar, the ratio of pixels printed perunit area, and the page feed position is detected based on thedetermination result. However, as the method of determining thecomplexity of performance, a position where bars having the same phrasesuccessively exist may also be detected, instead of the ratio of pixelsprinted per unit area in each bar. If the page feed position is set at aposition where bars having the same phrase continue, the player caneasily memorize the phrase and easily turn a page.

A second embodiment will explain a method of detecting a position wherebars having the same phrase continue. Whether bars having the samephrase continue can be determined by analyzing two successive bars.Needless to say, a phrase of three or more bars may also continue. Inthis case, it cannot be determined by only analyzing two successive barsthat a phrase of three or more bars continues. However, in terms of thecomplexity of performance, a position where a phrase of three or morebars continues is not suitable for a page feed position. This is becausefor a phrase of three or more bars, a bar having the same phrase as thatof a bar immediately before turn of a page does not exist in the nextpage, and it is difficult to memorize the bar. For this reason, it isdifficult to turn a page at a position where a phrase of three or morebars continues. This position is not suitable for a page feed position.From this, to detect a page feed position considering the complexity ofperformance, two successive bars are compared and analyzed by shiftingbars one by one.

FIG. 17 shows the sequence of concrete page feed position detectionprocessing according to the second embodiment. The respective steps ofthis processing will be explained. Steps up to step S1060, steps S1230to S1250, and step S1280 and subsequent steps are the same as those inFIG. 13, and a description thereof will not be repeated.

In step S1130, a counter N is initialized.

In step S1140, it is determined whether a bar of interest includes asign such as a G or F clef. If the bar of interest includes a sign, theprocess advance to step S1150; if it does not include any sign, to stepS1160.

In step S1150, the sign image is deleted from the bar image so as not todetermine that two bars, either of which includes a sign, have differentphrases upon analyzing the images of the two bars though the two barshave the same phrase.

In step S1160, the image data of the bar of interest is stored as Bar(N)in a memory in order to compare it with image data Bar(N+1) of the nextsuccessive bar in subsequent processing.

In step S1170, the counter N is incremented, and the process advances tostep S1180.

In step S1180, it is determined whether the value of the counter N istwo or more. If the value of the counter N is smaller than 2, theprocess returns to step S1140 to analyze the images of two successivebars. If the value of the counter N is equal to or larger than 2, theprocess advances to step S1190.

In step S1190, the absolute value of the difference between Bar(N) andBar(N+1) stored in the memory in step S1160 is obtained and saved asDiff in the memory.

In step S1200, it is determined whether the value Diff obtained in stepS1190 is equal to or smaller than a preset threshold Diff_(th). If thevalue Diff is equal to or smaller than the threshold Diff_(th), theprocess advances to step S1210; if it is larger than the thresholdDiff_(th), to step S1220. In this case, not whether Diff is 0, butwhether Diff is equal to or smaller than the threshold Diff_(th) isdetermined because even if bars have the same phrase, the bar width issometimes slightly different between bars. In this case, the positionsof notes may also slightly change. If the positions of notes evenslightly change, Diff does not become 0. For this reason, Diff iscompared with the threshold Diff_(th) to detect bars having the samephrase even when the bar width is slightly different between bars havingthe same phrase. In this case, the user may designate the thresholdDiff_(th). When the user sets the threshold Diff_(th) to a slightlylarge value, a position where two bars having not completely the samephrase but similar phrases continue can also be detected. Similarphrases can be easily memorized.

In step S1210, Bar(N+1) having Diff equal to or smaller than thethreshold Diff_(th) in step S1200 is set as a page feed positioncandidate, and the bar number BarNum of Bar(N+1) is stored in the memoryarea.

In step S1220, it is determined whether Bar(N+1) is the final bar of thegreat staff. If it is determined that Bar(N+1) is the final bar, theprocess advances to step S1240. If it is determined that Bar(N+1) is notthe final bar, the process advances to step S1230 to repeat steps S1130to S1210 for the next bar.

In step S1270, a bar number corresponding to a position closest to thebeginning or end of a page among bar numbers BarNum stored in the memoryin step S1210 is detected as a page feed position. This is because, evenif the page feed position is detected at a position near the center of apage, this position is not suitable for a page feed position, asdescribed in the first embodiment. Hence, a position as close aspossible to the beginning or end of a page is preferentially detected asa page feed position.

The method of detecting, as a page feed position, a position where barshaving the same phrase continue has been explained. However, such aposition may not exist in a page. Considering this case, when a positionwhere bars having the same phrase continue does not exist in a page,page feed position detection processing can be switched to detect a pagefeed position in the page according to the method of the firstembodiment.

The present invention is also practiced by supplying the program codesof software for implementing the functions of the above-describedembodiments to, for example, an apparatus connected to various devices,and operating these devices in accordance with programs stored in thecomputer of the apparatus or the like.

In this case, the program codes of the software implement the functionsof the above-described embodiments, and the program codes themselves anda means such as a storage medium for supplying the program codes to acomputer constitute the present invention. Concrete examples of thestorage medium are a flexible disk, hard disk, optical disk,magnetooptical disk, CD-ROM, magnetic tape, nonvolatile memory card, andROM.

The present invention includes program codes when the functions of theabove-described embodiments are implemented by executing the programcodes supplied to the computer. The present invention also includesprogram codes when the functions of the above-described embodiments areimplemented by an OS (Operating System) and another application softwareor the like running on the computer in cooperation with each other.

The functions of the above-described embodiments are also implementedwhen program codes supplied to the computer are stored in the memoriesof the function expansion board and function expansion unit of thecomputer, and, for example, the CPUs of the function expansion board andfunction expansion unit execute processing on the basis of theinstructions of the program codes. The present invention alsoincorporates these program codes.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2007-331072, filed Dec. 21, 2007, which is hereby incorporated byreference herein in its entirety.

1. A sheet music creation method of creating new sheet music by changinga page feed position in sheet music formed from a plurality of pages,the method comprising: inputting image data of the sheet music formedfrom the plurality of pages; dividing the image data input in the stepof inputting into bars; determining, for each page, complexity ofperformance of each bar divided in the steps of dividing; detecting aneasy-to-play portion as a page feed position for each page in accordancewith a determination result in the step of determining; changing alayout of the sheet music on the basis of the page feed positiondetected in the step of detecting; and outputting the sheet music whoselayout has been changed in the step of changing.
 2. The method accordingto claim 1, wherein the step of inputting inputs the image data of thesheet music which is formed from the plurality of pages and scanned by ascanner.
 3. The method according to claim 2, wherein when the page feedposition detected in the step of detecting exists in first half of sheetmusic of one page, the step of changing adds a portion before the pagefeed position in the sheet music of one page to an end of an immediatelypreceding page, and when the page feed position exists in second half,adds a portion after the page feed position in the sheet music of onepage to a beginning of a next page.
 4. The method according to claim 1,wherein the step of changing lays out a first bar of a next page at anend of sheet music of one page in a form different from a form ofremaining bars.
 5. The method according to claim 1, further comprisingthe step of selecting, from sheet music in which no page feed positionis changed, a page in which the page feed position is to be changed,wherein the step of determining determines the complexity of performanceof each bar in the page selected in the step of selecting, and the stepof detecting detects a page feed position in the page selected in thestep of selecting.
 6. The method according to claim 1, wherein the stepof outputting includes the step of printing layout-changed sheet musicby a printing apparatus.
 7. The method according to claim 1, wherein asfor the complexity of performance of each bar, as a ratio of pixelsprinted per unit area in an image range of each bar is higher, the stepof determining determines that the bar is more complex, and the step ofdetecting detects, as the page feed position, a position immediatelyafter a bar having a lowest ratio.
 8. The method according to claim 1,wherein as for the complexity of performance of each bar, the step ofdetermining determines that a portion at which bars having the samephrase continue is easy to play, and the step of detecting detects, asthe page feed position, a position between successive bars having thesame phrase.
 9. An image processing system comprising: a host apparatus;an image reading apparatus configured to be connected to the hostapparatus; and a printing apparatus configured to be connected to thehost apparatus; wherein the system is configured to create new sheetmusic by changing a page feed position in sheet music formed from aplurality of pages; wherein the image reading apparatus reads image dataof the sheet music formed from the plurality of pages; wherein the hostapparatus includes, a division unit configured to divide the image dataread by the image reading apparatus into bars; a determination unitconfigured to determine, for each page, complexity of performance ofeach bar divided by the division unit; a page feed position detectionunit configured to detect an easy-to-play portion as a page feedposition for each page in accordance with a determination result of thedetermination unit; and a layout change unit configured to change alayout of the sheet music on the basis of the page feed positiondetected by the page feed position detection unit; and wherein theprinting apparatus prints the sheet music whose layout has been changedby the layout change unit.
 10. An image processing system comprising: ahost apparatus; and a multifunctional peripheral configured to beconnected to the host apparatus, the multifunctional peripheralincluding, an image reading unit; and a printing unit; wherein thesystem is configured to create new sheet music by changing a page feedposition in sheet music formed from a plurality of pages; wherein thehost apparatus includes, a division unit configured to divide, intobars, image data of sheet music which is read by the image reading unitand formed from a plurality of pages; a determination unit configured todetermine, for each page, complexity of performance of each bar dividedby the division unit; a page feed position detection unit configured todetect an easy-to-play portion as a page feed position for each page inaccordance with a determination result of the determination unit; and alayout change unit configured to change a layout of the sheet music onthe basis of the page feed position detected by the page feed positiondetection unit; and wherein the multifunctional peripheral is configuredto cause the printing unit to print the sheet music whose layout hasbeen changed by the layout change unit.
 11. A computer-readable mediumcontaining computer-executable instructions to be utilized in an imageprocessing system for creating new sheet music by changing a page feedposition in sheet music formed from a plurality of pages, the mediumcomprising: computer-executable instructions for inputting image data ofthe sheet music formed from the plurality of pages; computer-executableinstructions for dividing the input image data into bars;computer-executable instructions for determining, for each page,complexity of performance of each divided bar; computer-executableinstructions for detecting an easy-to-play portion as a page feedposition for each page in accordance with a determination result fromthe determining step; computer-executable instructions for changing alayout of the sheet music on the basis of a detected page feed position;and computer-executable instructions for outputting the sheet musicwhose layout has been changed.